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ABSTRACT 

The  provision  of  comprehensive  services  by  a  complex 
modern  computer  installation  is  expensive.   In  the  face  of 
increasing  demand  for  computer  service,  system  expansion 
may  be  proposed.   This  expansion  may  not  be  necessary  if 
existing  resource  utilization  can  be  increased  or  more 
equally  distributed. 

This  research  investigates  the  possibility  of  increased 
system  throughput  through  a  balancing  of  the  demand  on  the 
individual  modules  of  an  IBM  2314  Disk  Facility.   The  per- 
formance of  the  disk  modules  is  measured  utilizing  a 
hardware  monitor.   The  hardware  monitor  is  also  used  to 
obtain  system  performance  profiles. 

Comparison  of  system  throughput  is  made  during  times 
when  different  sets  of  resources  are  avilable.   Recommendations 
are  made  to  improve  system  performance  by  rearranging  the 
data  sets  on  the  disk  modules » 
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I.   INTRODUCTION 

The  high  cost  of  providing  comprehensive  services  in  a 
modern  computer  installation  motivates  the  manager  to  reduce 
the  costs  or  at  least  maintain  costs  at  a  constant  level 
during  a  time  of  continually  increasing  demands  for  service. 
If  the  computer  system  is  not  completing  its  assigned  tasks 
according  to  the  required  schedule,  expansion  of  the  system 
capacity  may  be  proposed.   For  example,  additional  core 
storage  may  be  added,  a  faster,  greater  capacity  auxiliary 
storage  device  may  be  substituted  for  an  existing  device, 
faster  input/output  peripheral  devices  may  be  obtained,  or 
even  the  CPU  itself  may  be  upgraded.  All  of  these  alternatives 
involve  significant  financial  expenditure,  but  another 
alternative  exists  which  may  be  less  expensive.   That  is  to 
continue  to  utilize  the  same  equipment,  but  to  increase  the 
effective  utilization  of  this  equipment  to  meet  the  increased 
demand  for  computational  power » 

In  order  for  this  last  alternative  to  be  selected,  its 
feasibility  must  be  determined  and  before  making  this 
determination  one  must  measure  the  present  performance  of 
the  systemc 

The  measurement  of  the  performance  of  a  complex  computer 
system  is  difficult,  but  the  potential  rewards  are  significant 
and  well  documented  (References  1  and  2).   In  addition  to  the 
improvements  in  performance  made  possible  by  performance 


measurement,  the  measurements  provide  a  basis  for  future 
decisions  on  configuration  changes  and  system  expansiono 

This  research  is  directed  at  the  performance  measurement 
of  an  IBM  2314  Disk  Facility  and  its  associated  selector 
channels   The  performance  of  individual  disk  modules  is 
measured  and  the  resulting  data  is  analyzed „   Recommendations 
are  presented  to  improve  system  performance o 


II.   BACKGROUND  AND  OBJECTIVES 

Knowing  what  to  measure  with  a  hardware  monitor  is 
difficult.   If  measurements  are  too  gross,  specific  recommen- 
dations for  change  are  difficult  or  impossible  to  formulate, 
while  if  measurements  are  very  detailed  they  may  form  the 
basis  for  a  recommendation  to  improve  utilization  of  one 
component  of  a  system  without  considering  the  concurrent 
effects  on  the  overall  system  performance. 

This  research  is  part  of  a  continuing  effort  to  determine 
precisely  how  to  identify  and  measure  the  work  accomplished 
by  and  performance  of  a  complex  modern  computing  system. 
The  very  definitions  of  the  terms  "work",  "performance", 
and  "computer  power"  are  under  discussion  and  subject  to 
efforts  for  more  formal  definition  (Reference  3). 

Hanke  has  made  measurements  on  the  IBM  360  Model  67 
installed  at  the  Naval  Postgraduate  School  (Reference  4). 
He  reported  a  large  percentage  of  GPU  wait  only  time  (CPU 
in  wait  state  and  selector  channel  2  not  busy) »   One  possible 
cause  of  this  is  large  disk  arm  seek  time,  i.e.,  the  GPU 
and  selector  channel  are  both  waiting  for  the  disk  arm  to 
move  to  some  other  disk  track.   The  primary  objective  of  this 
research  was  to  measure  the  performance  of  the  IBM  2314 
Disk  Facility  and  its  associated  selector  channel  to  determine 
the  percentage  of  time  spent  by  the  GPU  and  selector  channel 
both  waiting  for  the  disk  arm  to  move  to  another  track,  then 


to  determine  if  this  arm  seek  time  accounted  for  the  majority 
of  the  CPU  wait  only  timco   In  addition  to  specific  measure- 
ments of  the  2314  Disk  Facility,  it  was  desirable  to  record 
broad  system  performance  profiles  to  determine  if  CPU  wait 
only  continued  to  be  significantly  high. 

Some  improvements  were  recommended  in  the  data  reduction 
and  analysis  programs  written  by  Hanke  (Reference  4).   These 
improvements  included  the  addition  of  the  ability  to  plot 
the  output  from  the  hardware  monitor  and  the  addition  of  a 
date  check  in  the  program  SMF  Grapho   Some  improvements  in 
the  statistical  analysis  of  data  to  determine  means,  variances 
and  correlations  was  also  required. 

Thus,  the  overall  objective  of  the  research  was  to 
combine  a  specific  performance  measurement  experiment  with 
supporting  data  reduction  and  analysis  in  order  to  make  a 
specific  recommendation  for  improvement  in  system  perfor- 
mance o   The  objective  of  this  thesis  is  to  report  the  results 
of  this  research  and  to  suggest  areas  for  further  research* 


III.   EXPERIMENTAL  PROCEDURE 

A„   MEASUREMENT  ENVIRONMENT 

The  computer  system  under  investigation  is  an  IBM  360 
Model  67  with  configuration  as  shown  in  Figure  I„   The 
system  was  operated  in  a  simplex  mode  (single  GPU)  with 
768K  bytes  of  core  storage  for  20  hours  per  weekday  and  as 
a  split  system  (separate  operating  systems  on  the  two  CPU's) 
for  four  hours  per  day.   On  Saturdays  and  Sundays  the  system 
is  run  from  0800-2000  in  a  simplex  mode.   While  a  split 
system  is  operating  from  1200  to  1600  each  weekday,  part  of 
the  computer  resources  are  assigned  to  a  time-sharing 
system,  GP/CMS  (Cambridge  Monitor  System).   The  major  cliange 
in  resources  available  for  batch  processing  operation  includes 
the  loss  of  256k  bytes  of  core  storage  and  the  2301  druai. 
Detailed  allocation  of  resources  during  the  four  hours  of 
time-sharing  is  shown  in  Figure  2. 

The  operating  system  under  investigation  is  OS/360  MVT 
(Multiprogramming  with  a  Variable  number  of  Tasks),   (The 
operation  of  the  CP/CMS  time-sharing  system  was  not  measured 
as  part  of  this  researcho)   During  the  twenty  hours  per  day 
without  time-sharing,  768K  bytes  of  core  storage  are  available 
with  478K  bytes  available  for  the  execution  of  problem 
programs  and  the  remaining  29 OK  bytes  for  use  by  the  operating 
system.   The  use  of  256K  bytes  by  the  time-sharing  system 
leaves  222K  bytes  for  problem  programs  during  the  1200-1600 


-•2 

<  to 

2  2 

en  -J 

£  2 


Naval  Postgraduate  School  IBM  360  Model  67 
Figure  1. 
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DEVICE  OS/MVT  GP/CMS 

CPU   2067-2  X 

CPU    2067-2  X 

PRINTER  KEYBOARD    1052-7  X 

PRINTER   KEYBOARD    1052-7  X 

CORE   STORAGE    2365-12  X 

CORE   STORAGE   2365-12  X 

CORE   STORAGE   2365-12  X 

DRUM  STORAGE    2301  ■  X 

DISK  STORAGE   2311    (8)  X 

DISK  STORAGE    2314  X 

TAPE  UNITS    2402-1  X  X 

CARD   READER    2501-B2  X" 

CARD   READ   PUNCH    2540  X 

PLOTTER    765    (2)  X 

PRINTER   1403 -Nl    (2)  X  X 

CHANNEL  CONTROLLER    2846-1    (2)  X  X 


Computer  Resource  Allocation  Under  CP/CMS 
Figure  2, 
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time  period.   (Since  this  research  was  completed  some  parts 
of  the  operating  system,  namely  the  resident  S\^C '  s ,  have  been 
made  non-resident  and  this  increased  the  usable  core  to  260K 
bytes  o) 

Operating  policy  also  varies  with  the  time  of  day.   The 
primary  objective  of  the  operating  policy  is  to  give  quick 
turnaround  for  small,  short  jobs  ( -^  lOOK  bytes,  ^20  seconds 
GPU  time).   No  particular  attempt  is  made  to  balance  the 
workload  of  the  system,  i.eo,  control  the  job  mix  to 
execute  both  I/O  bound  and  compute  bound  jobs  at  the  same 
time.   Control  of  the  job  mix  V70uld  be  difficult  as  job  entry 
is  by  way  of  a  user  operated  hot  card  reader.   Job  classes 
are  defined  to  give  the  highest  priority  to  the  small,  short 
jobs.   The  use  of  QUICKRUN  (Reference  5)  as  a  sub-system  of 
the  operating  system  is  also  highly  favorable  to  the  small 
jobs,  generally  providing  "instant"  turnaround  (less  than  five 
minutes)  for  the  small  jobs.   QUICKRUN  is  a  job  management 
system  which  processes  problem  programs  faster  than  OSAlVT 
by  reducing  the  operating  system  overhead,  associated  with 
each  jobo   Restrictions  on  jobs  eligible  to  be  run  under 
QUICKRUN  include  less  than  lOOK  bytes,  less  than  20  seconds 
of  CPU  time,  no  use  of  tape,  and  less  than  1000  lines  of 
printed  output. 

Job  arrivals  are  heavily  concentrated  in  the  afternoon 
with  the  peak  load  usually  coming  between  1^1-00-1600.   During 
the  month  of  March  1972  when  these  measurements  were  taken, 
24,500  jobs  were  processed;  of  these  11,700  were  under  QUICKRUN 
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B.   EXPERIMENTAL  DESCRIPTION 

The  primary  measuring  device  used  in  this  research  was 
the  Measurement  Engine,  a  hardware  monitor  manufactured  by 
Boole  and  Babbage,  Inc o o   The  use  of  the  Measurement  Engine 
in  system  performance  measurement  and  analysis  is  described 
in  References 4  and  60   The  Measurement  Engine  is  actually  a 
hardware  monitor  system  with  many  different  possible  con- 
figurations.  As  used  for  these  experiments,  the  configuration 
consisted  of  two  ME-1011  Event  Monitors  and  one  ME-2011  Paper 
Tape  Printer,  all  owned  by  this  institution.   Each  Event 
Monitor  can  receive  signals  from  eight  probes  attached  to  the 
host  computer.   The  probe  signals  may  then  be  combined  on  a 
user  wired  logic  plugboard  which  has  AND,  NOR,  INVERTER,  and 
FLIPFLOP  capabilities.   The  outputs  from  the  logic  plugboard 
are  then  routed  to  the  six  counters  and  the  paper  tape  printer 
Logic  signals  may  be  routed  betv/een  Event  Monitors  which  may 
be  stacked  one  upon  the  other » 

To  obtain  the  nine  signals  shown  in  Figure  3,  nine  probes 
were  connected  to  the  appropriate  computer  pins  also  shown 
in  Figure  3 ». 

SIGNAL  DEVICE  PIN 

GPU  manual  2067  EC2H4B09 

CPU  wait  2067  EC2J6B07 

Channel  2  busy  2360  BA3D6D04 

MVTREX  disk  arm  seek  2314  AA3H4D11* 

MVTLNX  disk  arm  seek  2314  AA3H4D11 

LINDA  disk  arm  seek  2314  AA3H4D11 

SPOOL  1  disk  arm  seek  2314  AA3H4D11 

SPOOL  2  disk  arm  seek  2314  AA3H4D11 

SPOOL  3  disk  arm  seek  2314  AA3H4D11 

*This  pin  is  probed  on  each  module  measured » 

Figure  3.   Hardware  Monitor  Signal  Probe  Connections 
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The  input  signals  were  combined  using  the  logic  board 
capability  of  the  Event  Monitor.   A  diagramatic  representation 
of  the  logic  is  shown  in  Figure  4o   The  resulting  signals 
representing  the  ten  events  shown  in  Figure  5  were  accumulated 
by  the  counters  of  the  tv^70  Event  Monitors  and  at  preselected 
time  intervals  were  recorded  by  the  Paper  Tape  Printer.   These 
paper  tape  data  were  then  keypunched  to  be  used  as  input 
to  the  program  Hardware  Graph  (Reference  4),  which  presents 
a  bar  graph  for  each  event  for  each  time  interval. 

1.  CPU  not  manual 

2o  CPU  wait 

3„  CPU  wait  and  selector  channel  2  busy 

4.  CPU  wait  and  selector  channel  2  not  busy 

5,  CPU  wait  and  selector  channel  2  nof  busy 
and  disk  module  MVTREX  arm  seeking 

6o   CPU  wait  and  selector  channel  2  not  busy 
and  disk  module  MVTLNX  arm  seeking 

7o   CPU  wait  and  selector  channel  2  not  busy 
and  disk  module  LINDA  arm  seeking 

8o   CPU  wait  and  selector  channel  2  not  busy 
and  disk  module  SPOOL  1  arm  seeking 

9o   CPU  wait  and  selector  channel  2  not  busy 
and  disk  module  SPOOL  2  arm  seeking 

10.   CPU  wait  and  selector  channel  2  not  busy 
and  disk  module  SPOOL  3  arm  seeking 

Events  Monitored. 
Figure  5. 

The  conditions  of  each  experiment  are  summarized  in 

Figure  6,  but  it  is  appropriate  here  to  discuss  some  of  the 

reasons  for  conducting  the  experiments  under  these  conditions. 
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Experiment 

Week  of 
quarter 

Day  of 
week 

Time  of 
day 

Size  of 
interval 

Number  oi 
intervals 

1 

10 

W 

1115- 
1715 

15  min. 

24 

2 

10 

V7 

1715- 
2315 

15  min. 

24 

3 

10 

Th 

0900- 
2100 

30  min. 

24 

h 

10 

F 

0900- 
2100 

30  min. 

24 

5 

10 

Sa 

0915- 
1915 

30  min. 

20 

6 

11 

Su 

0930- 
1930 

30  mino 

20 

7 

11 

Tu 

1015- 
2015 

60  min. 

10 

Experiment  Summary 
Figure  6. 
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In  order  to  insure  robustness  of  results,  it  was  desired 
to  conduct  worst  case  experiments  and  analysis.   The  tenth 
and  eleventh  weeks  of  a  twelve  week  academic  quarter  were 
chosen  as  appropriate  times  for  measurements  due  to  the 
historically  heavy  workload  during  these  t\"70  weeks. 

It  was  also  desired  to  compare  system  performance  during 
the  time  periods  when  the  time-sharing  system  CP/CMS  was 
being  utilized  against  the  periods  when  OS/MVT  was  operating 
exclusively.,   This  dictated  that  the  afternoon  be  included. 
Also,  the  highest  job  arrival  frequency  is  during  the 
afternoon. 

For  the  first  two  days'  experiments  (1  and  2),  a  time 
interval  of  15  minutes  was  chosen  in  order  to  determine  the 
range  of  values  over  relatively  short  time  intervals »   There 
were  no  wide  fluctuations  during  the  15  minute  intervals  so 
30  minute  intervals  were  chosen  for  the  remaining  experiments. 
The  50  minute  interval  was  chosen  for  the  final  experiment 
due  to  the  failure  of  the  paper  tape  printer.   The  hardware 
monitor  holds  the  accumulated  utilization  values  in  a  buffer 
for  output  to  the  paper  tape  printer  until  the  next  time 
interval  has  elapsed »   This  allows  the  experimenter  to  hand 
record  the  values  in  the  buffer  just  before  the  end  of  a  time 
interval  and  just  after  the  end  of  a  time  interval.   By 
recording  data  from  two  time  intervals,  the  experimenter 
may  then  by  physically  absent  from  the  hardware  monitor  for 
slightly  less  than  two  more  time  intervals.   For  example, 
by  using  the  60  minute  interval,  one  may  be  absent  from  the 
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hardware  monitor  for  about  1  hour  and  50  minutes  of  every 
2  hours  without  losing  any  data.   It  is  felt  that  these 
different  time  intervals  do  not  significantly  affect  the 
results  reported  herein. 

System  performance  was  monitored  for  a  total  of  66 
hours o   Of  this  time  there  were  768K  bytes  of  core  storage 
available  to  the  system  for  50  hours .   For  16  hours  512K 
bytes  of  core  storage  were  available  to  the  system  as  256K 
bytes  of  core  storage  and  the  2301  drum  were  being  utilized 
by  the  time-sharing  system.   The  66  hours  of  measurement 
time  were  divided  into  46  hours  during  weekdays  and  20  hours 
during  the  weekend. 
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IV.   DISCUSSION  OF  RESULTS 

A.   DISK  MODULE  PERFORMANCE 

The  six  modules  of  the  IBM  2314  Disk  Facility  whose 
performance  was  measured,  are  known  by  the  names  MVTREX ,  MVTLNX , 
LINDA,  SPOOL  1,  SPOOL  2,  and  SPOOL  3o   Two  other  user  disk 
modules  named  MARY  and  DUFFY  were  not  measured  because  their 
activity  is  much  lower  than  those  measured.   In  this  discussion 
the  comparisons  involve  the  condition  when  the  CPU  is  in  the 
wait  state  and  the  selector  channel  is  not  busy  and  a  disk 
arm  is  seeking  (moving  to  another  track).   This  condition 
will  be  referred  to,  for  example,  as  MVTREX  seek  without 
repeating  the  CPU  wait  and  channel  not  busy  qualifiers . 

System  performance  profiles  (Figure  7)  show  that  the 
CPU  wait  percentage  had  a  wide  range  of  variation  varying 
from  0  to  86  percent.   Averaged  over  the  seven  experiments 
the  mean  CPU  wait  was  51  percent.   The  CPU  wait  only  (CPU 
wait  and  selector  channel  2  not  busy)  averaged  over  the 
seven  experiments  ranged  from  6  to  55  percent  with  a  mean 
of  26  percento   Thus,  on  the  average,  the  CPU  is  idle  half 
the  time  and  of  this  CPU  idle  time  about  half  the  time  the 
channel  is  also  idle.   One  condition  that  may  cause  both 
the  CPU  and  channel  to  be  waiting  is  a  disk  arm  seeking 
(moving  to  another  track).   Data  from  the  seven  experiments 
showed  that  the  module  MVTLNX  had  more  arm  seek  time  than 
the  other  five  disk  modules  measured  (Figure  8),   The  ratio 
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of  MVTLNX  seek  to  the  other  disk  modules  ranged  from  1.9:1 
to  12o0:l„   The  ratio  of  MVTLNX  seek  to  the  mean  disk  seek 
was  2o65:l  averaged  over  the  seven  experiments.   Thus,  there 
was  an  unbalanced  demand  placed  on  this  one  disk  module, 
MVTLNX. 

What  then  were  the  contents  of  this  disk  module  which 
may  have  caused  this  imbalanced  demand?  MVTLNX  is  a  system 
module  with  three  particular  data  sets  of  interest.   The 
most  active  data  set  on  IWTLNX  was  the  operating  system  job 
queue o   This  job  queue  data  set  is  allocated  30  cylinders 
(4o3  million  bytes)  of  space  which  is  referenced  by  many 
parts  of  the  operating  system.   The  job  queue  must  be  accessed 
by  the  reader  program,  the  initiator  program,  the  writer 
program,  and  by  the  display  commands  issued  by  the 
operator's  console,  for  a  minimum  of  between  6  and  16 
accesses  per  jobo 

Another  significant  data  set  on  the  MVTLNX  module  is  the 
link  library o   This  data  set  is  allocated  50  cylinders  (7o2 
million  bytes)  of  spacco   The  link  library  contains  the 
executable  modules  for  the  reader  program,  the  writer  program., 
and  the  initiator  program,,   It  also  contains  the  language 
processing  modules  (FORTRAN  G,  FORTRAN  H,  PL/I ,  COBOL,  and 
RPG),  non-resident  operating  system  modules ,  supervisor  calls 
(SVC)  and  input/ output  error  recovery  modules.   This  data 
set  must  be  accessed  a  minimum  of  3  to  5  times  for  each  job 
executiono 

The  third  data  set  of  interest  in  the  module  MVTLNX  is 
used  for  recording  accounting  data.   System  Management 
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Facilities  (SMF)  information  is  written  into  this  third 
data  set.   SMF  is  an  optional  feature  of  0S/M7T  which 
records  system  and  job  performance  information.   (Use  of  SMF 
as  a  software  monitor  is  explained  by  Hanke  (Reference  4).) 
In  particular,  job  start  and  stop  times,  CPU  times  used  and 
identification  data  are  recorded  for  each  job  step  upon 
completion  of  the  job  step.   This  data  set  is  thus  accessed 
at  least  three  times  on  an  average,  non-QUIGKRUN  job  (once 
per  job  step) . 

This  one  disk  module  MVTLNX  therefore  contains  three 
data  sets  which  must  be  accessed  between  12-24  times  for  each 
job  execution.   This  would  be  the  case  for  a  typical  FORTRAN 
compile,  link-edit  and  execute  job,  which  account  for  about 
half  of  all  jobs  submitted,  not  including  those  FORTRAN  jobs 
run  under  QUICKRUNo 

The  questions  arise  as  to  which  of  the  data  sets  could 
be  transferred  from  MVTLNX  to  another  location,  where  the 
data  set  could  be  relocated,  and  what  effect  the  relocation 
would  have  on  system  performance.   The  first  data  set 
examined  was  the  job  queue »   The  job  queue  is  presently 
allocated  30  cylinders  (4.3  million  bytes)  of  space  v/ith  a 
resulting  capacity  of  about  150  jobs.   Assuming  that  the 
accesses  to  the  job  queue  account  for  between  50-67  percent 
of  the  disk  seek  activity  on  MVTLNX,  and  that  the  mean 
MVTLNX  seek  is  8.52  percent,  then  the  job  queue  seek  is  from 
0„50  *-  8.52  =  4.3  to  0.67  *  8.52  =  5.7  percent.   Considering 
that  20  hours  per  day  the  system  is  run  with  no  time-sharing 
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(i.eo,  with  the  230i  drum  available),  then  from  52  minutes 
to  69  minutes  (20  hours  *  0.057  =  I.IU  hours  =  69  minutes, 
20  hours  *  o043  =  .86  hours  =  51.6  minutes)  per  day  is  spent 
waiting  for  access  to  the  job  queue » 

Now  suppose  the  job  queue  were  placed  on  tlie  2301  drum. 
Arm  seek  delay  would  be  nonexistant  and  although  there  would 
be  some  delay  in  the  form  of  rotational  delay,  the  delay  would 
be  less  than  the  rotational  delay  of  the  2314  disk.   The 
improvement  gained  would  be  at  most  .86/20  hours  =  k,3 
percent  or  1.14  hours/20  hours  =5,7  percent.   The  2301  drum 
is  used  for  four  hours  per  day  in  support  of  the  GP/Cl-IS 
time-sharing  system  and  therefore  a  utility  program  would  be 
needed,  to  transfer  the  job  queue  from  the  2314  disk  to  the 
2301  drum  and  back  again  at  the  conclusion  of  the  time-sharing 
period.   This  transfer  of  the  job  queue  would  also  require 
a  reformatting  of  the  job  queue  to  coincide  with  the  recording 
techniques  used  on  the  2301  drum.   The  required  utility  pro- 
gram does  not  exist  and  one  NPS  system  programmer  suggested 
that  it  would  be  very  difficult  to  write.   Disadvantages 
in  moving  the  job  queue  from  the  disk  to  drum  and  back 
include  operator  inconvenience,  time  required  for  transfer, 
and  possible  error  and  subsequent  loss  of  the  job  queue. 

Another  alternative  would  be  to  move  the  job  queue  to 
another  disk  module.   Currently  there  would  have  to  be  an 
examination  of  the  other  disk  modules  to  determine  which 
data  sets  should  be  moved  to  make  room  for  the  job  queue, 
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as  there  is  not  sufficient  empty  space  on  the  other  modules 

to  relocate  the  job  queue   The  effect  on  performance  would 

be  difficult  to  estimate,  however,  this  task  would  have 

significant  value  of  serving  as  a  basis  for  comparison  with 

a  repeated  conduct  of  the  same  experiments  after  the  job 

queue  had  been  relocated.  (This  possibility  is  discussed  later.) 

What  other  data  set  might  be  moved?   The  link  library 
is  currently  allocated  50  cylinders  (7.2  million  bytes)  of 
space  which  is  about  twice  the  capacity  of  the  2301  drum. 
Similar  comments  to  those  about  moving  the  job  queue  to 
another  disk  module  apply  to  moving  the  link  library  to 
another  disk  module. 

This  leads  one  to  consider  the  System  Management  Facilities 
(SMF)  data  sets.   Two  data  sets,  SYSl.MANX  and  SYS1.K/\NY  are 
utilized  for  recording  SMF  data.   Two  data  sets  are  used  so 
that  when  one  data  set  is  full  the  recording  is  switched  to 
the  other  data  seto   Another  data  set  on  the  disk  module 
MVTLNX  is  named  SYSloSMFTUB.   The  data  from  SYSloMANX  or 
SYSl.MANY  is  tranf erred  to  SYSloSMFTUB  as  each  is  filled. 
Later  SYSl.SMFTUB  is  transferred  to  magnetic  tapco   Wlien  the 
transfers  from  SYSl.MANX  or  SYSl.MANY  to  SYSloSMFTUB  take 
place,  the  disk  arm  must  move  back  and  forth  on  the  same 
disk  module,  the  same  disk  module  which  already  is  tlie  most 
active.   This  occurs  about  once  per  day  and  the  transfer  is 
usually  done  on  the  0000-0800  shift  to  minimize  the  effect 
of  disk  arm  interference  on  system  performance. 

Assuming  that  the  SMF  recording  is  12 » 5  to  25  percent  of 
the  activity  on  the  disk  module  MVTLNX  and  using  the  mean  of 
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8o52  percent  MVTLNX  seek  averaged  over  the  seven  experiments, 
SMF  recording  would  account  for  .25  *   80 52  =  2. 13  percent  of 
MVTLNX  seek.   Taking  2. 15  percent  *  20  hours  =  0.430  hours 
=25.8  minutes  per  day  spent  waiting  for  the  disk  arm  to  move 
to  another  track  in  order  to  record  SMF  datao   Considering 
that  this  time  also  causes  contention  with  the  job  queue  and 
link  library  activity,  it  would  be  advantageous  to  record 
SMF  data  on  one  of  the  more  lightly  used  disk  modules o 
Elimination  of  the  25 „ 8  minutes  SMF  time  would  represent 
at  most  .43  hours/20  hours  =2.15  percent  improvement  in 
the  activity,  performance,  or  lo07  percent  if  SMF  recording 
is  12.5  percent. 

If  the  two  suggested  changes  in  the  contents  of  the 
disk  module  IWTLNX  were  made,  moving  the  job  queue  to  the 
2301  drum  and  moving  the  SMF   data  sets  to  a  less  active  disk 
module,  the  total  improvement  would  be  at  best  5.7  percent  + 
2ol5  percent  =  7.85  percent  improvement.   Using  an  average 
of  44  job  steps  executed  per  hour  this  7.85  percent  improve- 
ment would  represent  3,45  additional  job  steps  per  hour 
throughput  or  69  additional  job  steps  per  20  hour  day. 

Two  implied  assumptions  affecting  disk  seek  time  that 
should  be  explained  here  are  the  order  of  requests  to  the 
disk  and  the  distance  between  active  data  sets.   Since  the 
exact  order  of  requests  is  unknown  and  the  requests  do  not 
follow  any  fixed  pattern  in  a  multiprogramming  environment, 
the  assumption  of  random  ordering  seems  reasonable.   The 
three  critical  data  sets  -  SYSl.JOBQUE,  SYSI.LINICLIB ,  and 
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SYSl.MANX(Y)  -  are  Located  contiguously  so  as  to  minimize 
the  arm  seeking  delay  and  thus  neglecting  the  actual  distance 
moved  and  averaging  the  arm  seek  times  seems  reasonable. 

Bo   SYSTEM  TtlROUGHPUT 

Discussion  of  disk  performance  in  particular  and  computer 
system  performance  in  general  must  be  considered  in  the 
context  of  system  throughput o   During  the  month  of  March  1972 
when  these  experiments  were  performed,  the  computer  center 
processed  24,497  jobs.   Of  this  total  number  of  jobs,  11,681 
were  run  under  the  QUICKRUN  job  aianagement  system.   Figure  9 
shows  the  system  throughput  in  terms  of  jobs  completed  during 
each  hour  of  the  day.   Considering  the  period  of  time  from 
1200-1600  the  average  number  of  jobs  completed  per  hour  is 
1933  whereas  for  the  next  busiest  four  hours  (1000-1200  and 
1600-1800)  the  average  number  of  jobs  completed  is  1699. 
If  these  averages  are  normalized  to  reflect  the  different 
quantities  of  problem  program  core  available  (222K  bytes 
from  1200-1600  and  478K  bytes  from  1000-1200  and  1600-1800), 
then  the  throughput  per  unit  core  is  even  greater  during  the 
1200-1600  time  period  while  256K  bytes  core  storage  are  lost 
to  the  time-sharing  systemo   Lest  one  conclude  that  a  reduced 
amount  of  core  storage  improves  system  throughput  one  must 
consider  the  different  operating  policies  in  effect  during 
these  two  different  time  periods ^ 

During  the  1200-1600  time  period  when  only  222K  bytes 
of  core  storage  are  available,  only  small,  short  jobs,  lOOK 
bytes  or  less, 20  seconds  CPU  time  or  less,  are  allowed  to  be 
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Hour  Ending 
at  Time 

Jobs 
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0100 

434 
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1900 
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4.6 

0800 
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0.3 

2000 
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4,2 

0900 
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2.3 
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1139 

4,6 
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5,4 

2200 

1261 

5.1 

1100 

1749 

7.1 

2300 

1153 

4.7 
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1495 

6,1 

2400 

946 

3.9 

TOTAL 

24,497 

100% 

QUICKRUN 

11,681 

47 

,5% 

March  1972  System  Throughput 
Figure  9, 


28 


run.   This  control  is  obtained  by  a  combination  of  two 
factors o   First,  job  classes  are  defined  to  segregate  these 
jobs  into  one  class  and  secondly,  the  operator  controls  the 
starting  of  initiator  programs  to  run  only  this  one  class  of 
jobs.   Thus,  the  operating  policy  favors  the  predominant 
job  type,  giving  fast  turnaround  to  these  jobs  and  operating 
within  the  core  storage  limitations  imposed  by  the  loss  of 
256K  bytes  of  core  storage  for  use  by  the  time-sharing 
system.   This  operating  policy  discriminates  against  larger, 
longer  jobs  and  also  has  an  effect  on  system  utilization. 
There  is  not  a  mix  of  I/O  bound  jobs  and  compute  bound  jobs 
during  this  time  period  so  that  CPU  utilization  decreases 
while  1/0  activity  increases  (Figure  10). 

C.   PLOTTING  AND  STATISTICAL  ANALYSIS  PROGRAMS 

Some  improvements  were  recommended  in  the  data  reduction 
and  analysis  programs  written  by  Hanke  (Reference  4).   A 
program.  Hardware  Graph,  processes  data  from  the  hardware 
monitor  by  reading  keypunched  data  cards  and  producing  bar 
graphs  for  each  event  monitored.   It  was  desired  to  plot 
multiple  events  on  one  graph  so  that  the  analyst  might  be 
able  to  determine  trends  or  possible  interaction  between 
various  events.   The  plotting  program  listed  in  Appendix  A 
is  adapted  from  the  locally  obtained  program  STPLOTo   By 
changing  a  FORTRAN  READ  statement  and  corresponding  FORMAT 
statement,  the  user  may  plot  various  combinations  of  events, 
up  to  a  maximum  of  ten.   The  plot  is  output  on  the  line 
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512K  bytes   768K  bytes 
EVENT  no  drum    and  drum    Ratios 

CPU  wait  65ol8       47.13       1„38 

CPU  wait  and  channel 
not  busy 

CPU  wait  and  channel  busy 

CPU  wait,  and  channel  not 
busy  and  MVTREX  seek 

CPU  wait  and  channel  not 
busy  and  MVTLNX  seek 

CPU  wait  and  channel  not 
busy  and  LINDA  seek 

CPU  wait  and  channel  not 
busy  and  SPOOL  1  seek 

CPU  wait  and  channel  not 
busy  and  SPOOL  2  seek 

CPU  wait  and  channel  not 
busy  and  SPOOL  3  seek 


Comparison  of  OS/MVT  performance 
with  768K  bytes  vs o  5I2K  bytes  of  Core 
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printer  and  the  user  must  draw  lines  to  connect  the  points 
corresponding  to  the  events  plotted.   The  rapid  turnaround 
for  this  program  makes  it  very  useful  for  quick  visual 
analysis  of  experimental  results. 

Hanke's  program,  SMF  Graph,  reads  the  System  Management 
Facilities  data  from  the  SMF  data  set  on  the  disk  module 
MVTLNX  and  provides  a  summary  and  some  analysis  of  this 
job  stream  data.   One  input  parameter  to  this  program  is 
time  of  day  when  measurement  starts.   This  is  adequate  to 
locate  the  desired  SMF  data  if  data  from  only  one  day  is 
currently  recorded.   Sometimes  data  from  more  than  one  day 
is  in  the  SMF  data  set  in  v/hich  case  the  desired  data  might 
not  be  obtained  using  the  original  version  of  SMF  Graph. 
An  assembly  language  subprogram  was  added  to  SMF  Graph  to 
require  the  user  to  input  the  desired  date  as  another  input 
parameter  to  Sl^  Graph  and  to  give  SMF  Graph  the  capability 
to  check  for  that  date  in  the  SMF  data  set. 

Statistical  analysis  of  the  hardv/are  monitor  output  v/as 
performed  with  the  assistance  of  programs  from  UCLA's  BIMED 
series  (Reference  7).  These  programs  provide  many  standard 
statistical  measures  such  as  means,  variances,  correlations 
with  a  minimum  effort  on  the  part  of  the  user.  An  example 
of  the  results  of  computation  for  one  experiment  is  shown 
in  Appendix  C. 

D.   FIGURE  OF  MERIT 

During  the  course  of  this  research,  the  question  arose 
as  to  whether  the  results  obtained  were  typical  of  those 


31 


which  might  be  obtained  by  similar  experiments  on  other 
computer  systems.   Also  what  experiments  in  measuring  computer 
performance  are  in  progress  at  other  university  computing 
centers?   Estrin  in  Reference  8  states  that  the  results  of 
experiments  should  be  reproducible  in  order  to  be  of  any 
value  for  subsequent  generalization. 

For  these  and  other  reasons,  a  survey  was  designed  to 
inquire  about  the  computer  performance  at  other  computer 
facilities o   Shown  in  Appendix  D,  this  survey  will  be  sent 
to  many  installations  which  use  an  IBM  360/67  and  to  many 
other  universities*   The  results  v/ill  be  compiled  and  made 
available  to  contributors  in  an  effort  tov/ard  further 
understanding  of  computer  performance  measurement  and 
computer  system  performaince  optimization. 

One  key  question  in  the  survey  asks,  "Is  there  any  one 
overall  figure  of  merit  or  performance  index  computed  by 
combination  of  several  performance  parameters?   (Please  give 
formula)".   The  possibility  of  obtaining  a  concise  answer 
to  this  question  seems  sufficiently  remote  since  very  little 
research  has  been  done  on  this  problem,  although  this 
question  is  currently  under  study  at  this  institution..   If 
there  is  a  valid  figure  of  merit  for  a  computer  installation, 
or  a  computer  operating  environment,  it  would  certainly  be 
of  interest  and  of  value  to  other  computer  center  staffs. 
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V.   CONCLUSIONS  AND  RECOMMENDATIONS 

There  are  three  positive  results  derived  from  the  conduct 
of  this  research.   First,  the  actual  performance  of  the 
computer  system  during  a  stipulated  time  period  can  be  stated 
as  a  fact  rather  than  a  conjecture;  this  can  be  used  as  a 
basis  for  future  performance  comparisons »   Secondly,  a 
positive  recommendation  for  improvement  can  be  made  and 
thirdly,  the  author  is  now  prepared  to  conduct  further  per- 
formance evaluation  analyses  of  computer  systems . 

The  ability  to  state  the  performance  of  a  computer  system 
as  a  fact  is  valuable  to  the  manager  of  a  computer  system. 
Plans  and  decisions  can  be  based  on  this  factual  performance 
data  with  some  level  of  confidence,  which  is  certainly  greater 
than  the  confidence  based  on  unproven  conjectures  »   In 
addition,  future  performance  measurements  can  use  the  results 
reported  here  as  a  basis  for  comparison.   Anj^  comparison, 
however,  would  have  to  carefully  reconsider  the  measurement 
environment o 

The  ability  to  make  a  positive  recommendation  is 
particularly  significant.   It  may  be  very  interesting  to 
measure  performance  of  various  components  of  a  computer 
system,  however  if  no  positive  recommendation  for  improve- 
ment can  be  made  the  effort  expended  in  measurement  is  wasted. 
The  recommendation  from  this  research  is  to  move  the 
operating  system  job  queue  data  set  and  the  System  Management 
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Facility  (SMF)  recording  data  set  to  a  more  Lightly  used 
disk  module  on  the  2314  Disk  Facility,   Using  three  disk 
modules  for  storage  of  operating  system  data  sets  would 
balance  the  demand  on  the  individual  disk  modules.   Moving 
the  job  queue  to  a  third  system  disk  module  could  lower  the 
mean  seek  wait  on  MVTLNX  by  2.82  percent  (8„52  -  5.7)  (Section 
IV,  A).   This  would  result  in  a  2o9  percent  increase  in 
system  throughput  (2 o 82/100-2 .82)  plus  some  additional  increase 
due  to  the  elimination  of  arm  seek  contention  on  the  disk 
module  MVTLNX.   Balancing  the  demand  on  the  individual  disk 
modules  is  therefore  estimated  to  represent  a  3  to  5  percent 
improvement  in  system  throughput. 

The  computer  system  at  the  Naval  Postgraduate  School  is 
owned  by  the  Uo  S,  Navy.   Using  the  replacement  cost  of 
$4o8  million  and  an  estimated  60  months  (5  years)  of  system 
life,  a  monthly  lease  cost  of  (4.8  million/60  months)  $80,000 
may  be  assumed.  A  3  to  5  percent  improvement  thus  represents 
a  $2400  to  $4000  potential  savings.  A  $2400  to  $4000  monthly 
savings  would  pay  for  the  cost  of  the  hardv7are  monitor  used 
for  the  performance  measurements  in  less,  than  4  to  8  months 
timeo  Thus,  this  one  experiment  in  performance  improvement, 
by  paying  for  the  hardware  monitor,  provides  the  potential 
for  future  performance  measurement  efforts  at  essentially 
no  cos to 

Concurrent  with  the  reporting  of  this  research,  a  later 
version  of  the  operating  system  knov/n  as  Release  20  of 
OS/MVT  is  being  implemented  at  this  computer  center.   A 
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decision  has  been  made  to  eliminate  the  use  of  the  disk 
module  name  LINDA  as  a  user  disk  module  and  to  use  LINDA 
as  a  third  operating  system  disk  module.   Thus,  the  results 
of  performance  measurement  are  providing  an  input  to  the 
decision  making  process  for  configuration  changes.   It  is 
important  to  suggest  that  measurements  be  taken  to  verify 
the  suggested  improvement  in  system  performance  and  to 
determine  if  the  new  version  of  the  operating  system  has 
created  any  previously  unkno\<rn  problems. 

The  preparation  and  education  of  the  author  to  conduct 
future  performance  evaluation  analyses  of  computer  systems 
is  a  result  of  this  research  effort  which  may  be  of  real 
benefit  to  the  Navyo   The  number  of  trained  analysts  in 
computer  system  performance  evaluation  is  small  in  contrast 
to  a  growing  need.   It  does  not  appear  that  main  frame 
manufacturers  are  going  to  expend  great  effort  to  assist 
clients  in  performance  optimization  through  performance 
measurement  as  this  would  probably  reduce  sales  of  additional 
equipment o   The  users  therefore  v/ill  have  to  train  their 
own  performance  analysts  or  resort  to  outside  consultants 
in  order  to  use  performance  measurement  to  optimize  system 
resource  utilization. 

Further  performance  measurement  of  the  computer  system 

at  this  installation  would  be  useful.   Questions  requiring 

further  research  include: 

lo   What  part  of  the  GPU  v/ait  only  time  is  spent 
vjaiting  for  an  operator's  console  response. 
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2,  Does  the  operator's  console  activity  vary 
widely  from  shift  to  shift? 

3.  How  has  the  addition  of  the  IBM  2321  Data 
Cell  affected  system  performance? 

4.  What  is  the  effect  of  having  non-resident 
Supervisor  Calls  (SVG's)  when  only  512K 
bytes  of  core  storage  is  available? 

5,  V7hat  other  parts  of  the  operating  system 
could  be  made  non-resident? 

In  addition  to  performing  specific  performance  measurement 
experiments,  it  is  recommended  that  this  computer  installation 
establish  a  plan  for  periodic  system  profile  measurements » 
Monthly  accounting  data  is  currently  recorded  and  presented 
to  the  analyst  in  a  very  usable  form.   The  same  amount  of 
effort- should  be  expended  to  provide  monthly  hardware  per- 
formance profile  information  to  accompany  the  accounting  data. 

This  thesis  describes  the  steps  taken  to  improve  the 
performance  of  a  computer  system.   Further  improvements  in 
performance  may  be  available  for  the  cost  of  performing 
further  analysis.   Since  each  one  and  a  half  percent  improve- 
ment amounts  to  $1200  per  month  increase  in  computing  pov;er 
for  the  rest  of  the  system  life,  these  improvements  should 
be  actively  pursued. 
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APPENDIX  A 
PLOTTING  PROGRAM 
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APPENDIX   G 
CORRELATION  TUTRICES 
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APPENDIX  D 
FIGURE  OF  MERIT  SURVEY 

NAVAL  POSTGRADUATE  SCHOOL 
Monterey,  California 


Dear  Sirs : 

As  part  of  a  continuing  study  of  computer  performance 
measur-ement ,  a  survey  is  being  undertaken  by  the  Naval 
Postgraduate  School  "Computer  Science  Group".   This  survey 
will  seek  to  collect  information  about  performance  measure- 
ment at  other  computer  installations.   We  are  most  interested 
in  how  performance  is  measured  (hardware  monitors,  softv/are 
monitors,  accounting  data),  what  parameters  are  measured 
(CPU  utilization,  I/O  overlap,  core  utilization),  v;hat 
typical  or  realistic  values  for  these  parameters  for 
particular  job  streams,  and  ver>^  importantly,  what  use  is 
made  of  these  results. 

Your  cooperation  is  requested  in  completing  the  enclosed 
form  as  completely  and  accurately  as  possible.   The  results 
of  all  returned  surveys  will  be  compiled  and  distributed 
to  all  contributors o 

Sincerely, 


G.  H.  SYMS 
Assistant  Professor 
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Installation  Naiae  Point  of  Contact 


CD   Main  Frame  Designation/Model       T^l   Ovm/Leas e/Rent 


(3)   Disk  Units  Cmodel)  T^TTn umber  C5)Tape  Drives  C6)Number  hours 

of  operation/day 


C7)  Gore  Storage  (,8)   Amount  (9;  Bulk  (slow;  Core  Stg.  CIO;  Amount 


(11)  Drum  (12)  Capacity  (13)  Terminals  C14)NumbGr  (15)  Oper. 

(time  sliare)  Systems 


(16)  Printers  (17)  Card  Reader/Punch  (18)  Other  Input/Ouput "Devic es 
19«   Size  of  user  community?   (Students,  faculty,  staff) 

20o   Job  stream 

a.   Jobs /month: 


bo   Job  size  distribution  (core  used): 


c.   Job  time  distribution  (CPU  time  used): 

21 o   Turnaround  time 

a.  Average  per  job:  

b.  Distribution:   

22 o   What  type  of  performance  measurements  are  implemented? 


kk 


ao  Hardware  monitor? 

1)  Model: 

2)  Own/Lease: 

3)  Configuration: 
NOo  probes: 

No.  accumulators: 

(counters) 
Recording  media: 

(mag  tape,  paper  tape) 

b.  Software  monitor? 

1 )  Name : 

2)  Own/Lease: 

3)  Capabilities: 

c.  Accounting  routines 

1)  Acquired  from  manufacturer/locally  developed 

23 o  Are  the  outputs  of  any  measurement  tools  used  as  inputs 

to  any  type  of  configuration  simulations?   If  so,  v/hich  ones? 

24.  Just  what  parameters  are  measured  in  detail?  Please  give 
yes  or  no  and  recent  mean  values  or  ranges  if  possible. 

a.  CPU  utilization 

b.  Channel  utilization 

c.  Channel  utilization  while  CPU  wait 

d.  Device  utilization 

transfer 

seek 

queue  length 

e.  Length  of  job  queue 

maximum 
average 

f.  Core  segment  utilization 
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g.   Overhead  time  (%) 

ho   System  Data  Sets 

Trans  f er 

Seek 

Queue  length 

i.   Supervisor  Galls 

Active 

Loading 

Inactive 

Job  Stream  Data 

j.   Job  arrival  distribution 

k.   Distribution  of  jobs  by  language 

lo   Distribution  of  jobs  by  core  size  request 

m.   Distribution  of  jobs  by  time  requested 

n.   Distribution  of  turnaround  time  by  job  time 

Oo  Distribution  of  turnaround  time  by  job  size  (core) 

Po   Amount  of  I/O  per  job 

q.  "Cost"  per  job  (or  charge  schedule) 

25.  Is  a  full  time  time-sharing  system  supported? 

What  system? 

26 o  If  only  a  part  time  time-sharing  is  supported,  during 
v/hat  hours  of  the  day  is  it  available? 

27 o  Is  a  remote  job  entry  capability  supported? 

28 o  Can  the  user  monitor  the  queue  status  to  determine 
where  his  job  is  located? 

29 o  Are  your "customers /users"  satisfied  with  the  performance 
of  your  computer  system? 

30.  How  do  you  know? 

31.  Are  the  staff /operators  satisfied  with  the  performance 
of  computer  system? 
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32 .   How  do  you  know? 

33 o  Is  there  any  one  overall  figure  of  merit  or  performance 
index  computered  by  combination  of  several  performance 
parameters?   (Please  give  formula.) 

34o   Which  parameters  in  question  2k   do  you  consider  most 
significant  as  an  indication  of  computer  system 
performance? 
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3.    ABSTRACT 


The  provision  of  comprehensive  services  by  a  complex 
modern  computer  installation  is  expensive.   In  the  face  of 
increasing  demand  for  computer  service,  system  expansion 
may  be  proposed o   This  e^!:pansion  may  not  be  necessary  if 
existing  resource  utilization  can  be  increased  or  more 
equally  distributed » 

This  research  investigates  the  possibility  of  increased 
system  throughput  through  a  balancing  of  the  demand  on  the 
individual  modules  of  an  IBM  2314  Disk  Facility.   The  per- 
formance of  the  disk  modules  is  measured  utilizing  a 
hardware  monitor.   The  hardware  monitor  is  also  used  to 
obtain  system  performance  profiles. 

Comparison  of  system  throughput  is  made  during  times 
when  different  sets  of  resources  are  available.   Recommendations 
are  made  to  improve  system  performance  by  rearranging  the 
data  sets  on  the  disk  modules. 
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